Recurrent Memory Networks for Language Modeling

Recurrent Neural Networks (RNN) have obtained excellent result in many natural language processing (NLP) tasks. However, understanding and interpreting the source of this success remains a challenge. In this paper, we propose Recurrent Memory Network (RMN), a novel RNN architecture, that not only amplifies the power of RNN but also facilitates our understanding of its internal functioning and allows us to discover underlying patterns in data. We demonstrate the power of RMN on language modeling and sentence completion tasks. On language modeling, RMN outperforms Long Short-Term Memory (LSTM) network on three large German, Italian, and English dataset. Additionally we perform in-depth analysis of various linguistic dimensions that RMN captures. On Sentence Completion Challenge, for which it is essential to capture sentence coherence, our RMN obtains 69.2% accuracy, surpassing the previous state-of-the-art by a large margin.Comment: 8 pages, 6 figures. Accepted at NAACL 201

Unsupervised Neural Hidden Markov Models

In this work, we present the first results for neuralizing an Unsupervised Hidden Markov Model. We evaluate our approach on tag in- duction. Our approach outperforms existing generative models and is competitive with the state-of-the-art though with a simpler model easily extended to include additional context.Comment: accepted at EMNLP 2016, Workshop on Structured Prediction for NLP. Oral presentatio

Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations

We investigate whether off-the-shelf deep bidirectional sentence representations trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This approach only leverages a mix of monolingual corpora in many languages and does not require any translation data making it applicable to low-resource languages. In our experiments we outperform the best CoNLL 2018 language-specific systems in all of the shared task's six truly low-resource languages while using a single system. However, we also find that (i) parsing accuracy still varies dramatically when changing the training languages and (ii) in some target languages zero-shot transfer fails under all tested conditions, raising concerns on the 'universality' of the whole approach.Comment: DeepLo workshop, EMNLP 201

Viability of Non-Coplanar VMAT for Liver SBRT as Compared to Coplanar VMAT and Beam Orientation Optimized 4π IMRT.

PurposeThe 4π static non-coplanar radiotherapy delivery technique has demonstrated better normal tissue sparing and dose conformity than the clinically used volumetric modulated arc therapy (VMAT). It is unclear whether this is a fundamental limitation of VMAT delivery or the coplanar nature of its typical clinical plans. The dosimetry and the limits of normal tissue toxicity constrained dose escalation of coplanar VMAT, non-coplanar VMAT and 4π radiotherapy are quantified in this study.Methods and materialsClinical stereotactic body radiation therapy plans for 20 liver patients receiving 30-60 Gy using coplanar VMAT (cVMAT) were re-planned using 3-4 partial non-coplanar arcs (nVMAT) and 4π with 20 intensity-modulated non-coplanar fields. The conformity number (CN), homogeneity index (HI), 50% dose spillage volume (R50), normal liver volume receiving >15 Gy (VL>15), dose to organs at risk (OARs), and tumor control probability (TCP) were compared for all three treatment plans. The maximum tolerable dose (MTD) yielding a normal liver normal tissue control probability (NTCP) below 1%, 5%, and 10% was calculated with the Lyman-Kutcher-Burman model for each plan, as well as the resulting survival fractions at one, two, three, and four years.ResultsCompared to cVMAT, the nVMAT and 4π plans reduced VL>15 by an average of 5 cm3 and 80 cm3, respectively. 4π reduced the 50% dose spillage volume by ~23% compared to both VMAT plans, and either significantly decreased or maintained OAR doses. The 4π MTDs and survival fractions were significantly higher than both cVMAT and nVMAT (p<0.05) for all normal liver NTCP limits used in this study.ConclusionsThe 4π technique provides significantly better OAR sparing than both cVMAT and vMAT and enables more clinically relevant dose escalation for tumor local control. Therefore, despite the current accessibility of nVMAT, it is not a viable alternative to 4π for liver SBRT